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BACKGROUND OF THE INVENTION 

1. FIELD OF THE INVENTION: 

The present invention relates to a signal processing 
5 apparatus for processing an acoustic signal reproduced 
together with an image signal and a recording medium, and 
specifically to a signal processing signal for providing 
a viewer/ listener with a perception of distance of an 
acoustic image matched to the situation represented hy an 
10 image signal reproduced and thus realizing a viewing and 
listening environment in which image data and acoustic data 
match each other, and a recording medium having such image 
data and acoustic data recorded thereon . 

15 2. DESCRIPTION OF THE RELATED ART: 

Recently, optical disks such as laser disks and DVDs 
(digital versatile disks) have been widely used as recording 
media for storing acoustic data together with image data, 
in addition to video tapes. More and more households are 

20 now provided with an environment for allowing people to 
easily enjoy an audio and visual experience, using laser 
disk players and DVD players for reproducing data stored 
in the laser disks and DVDs, In addition, standards such 
as MPEG allows acoustic data and image data to he compressed 

25 together, so that individual viewers/listeners can enjoy 
audio and video reproduction on a personal computer. 

In general, under such a reproduction environment, 
however, the image data and the acoustic data are not 
30 sufficiently matched together. For example, while an image 
changes from a short range view to a distant view or pans 
outs to the left or to the right, an acoustic image is fixed 
at a certain position. 



- 2 - 



P26156 



In order to solve such a problem and thus provide 
the viewer/ listener with an improved environment of viewing 
and listening reproduction of image data and acoustic data, 
5 various proposals have been made. 



pi . 



For example, Japanese Laid-Open Publication 
No. 9-70094 discloses a technology for installing a sensor 
for detecting a motion of the head of a viewer/listener and 
10 correcting the acoustic signal based on an output signal 
from the sensor, so as to change the position of the acoustic 
image to match the motion of the head of the viewer/listener. 

International Publication W095/22235 discloses a 
15 technology for installing a sensor for detecting a motion 
of the head of a viewer/ listener and perf orming sound source 
localization control in synchronization with the video. 

However, conventional signal processing 
20 apparatuses using the above -described technologies can only 
use a filter prepared in the signal processing apparatuses 
in order to correct the acoustic signal. Therefore, it is 
impossible to correct the acoustic signal as desired by the 
viewer/listener or to reflect the intent of the content 
25 producer to correct the acoustic signal. 

Even when a filter for correcting the acoustic 
signal as desired by the viewer/ listener is prepared, the 
acoustic signal used needs to be corrected by a personal 
30 computer or the like. Therefore, in order to guarantee the 
matching of the image data and the acoustic data, extensive 
work is required. 
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A plurality of signal processing methods have been 
proposed for realizing movement of an acoustic image on a 
monitor screen for displaying an image. No signal 
processing method for providing the viewer/listener with 
a perception of distance of an acoustic image (depth of the 
acoustic image) with a small memory capacity and a small 
calculation amount has been proposed. 

SUMMARY OF THE INVENTION 

According to one aspect of the invention, a signal 
processing apparatus for processing an acoustic signal 
reproduced together with an image signal includes a memory 
for storing a plurality of filter coefficients for 
correcting the acoustic signal; a filter coefficient 
selection section for receiving a correction command, from 
outside the signal processing apparatus, for specifying a 
correction method for the acoustic signal and selecting at 
least one of the plurality of filter coefficients stored 
in the memory based on the correction command; and a 
correction section for correcting the acoustic signal using 
the at least one filter coefficient selected by the filter 
coefficient selection section . 

Due to such a structure, a signal processing 
apparatus according to the present invention allows the 
correction method of an acoustic signal to be changed in 
accordance with the change in an image signal or an acoustic 
signal. Thus, the viewer/listener can receive, through a 
speaker or headphones, a sound matching an image being 
displayed by an image display apparatus. As a result, the 
viewer/listener does not notice any discrepancies in a 
relationship between the image and the sound. 
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Also due to such a structure, a signal processing 
apparatus according to the present invention allows the 
correction method of an acoustic signal to be changed in 
5 accordance with the acoustic characteristic of the speaker 
or the headphones used by the viewer/listener or the acoustic 
characteristic based on the individual body features, for 
example, the shape of the ears and the face of the 
viewer/listener. As a result, a more favorable listening 
10 environment can be provided to the viewer /listener . 

Since the filter coefficients are stored in the 
memory, it is not necessary to receive the filter 
coefficients from outside the signal processing apparatus 

15 while the image signal and the acoustic signal are being 
reproduced. Accordingly, when the signal processing 
apparatus receives a correction command, the filter 
coefficients can be switched more frequently in accordance 
with the change in the image signal and the acoustic signal. 

20 As a result, the correction method for the acoustic signal 
can be changed while reflecting the intent of the producer 
of the image signal and the acoustic signal (contents). 

In one embodiment of the invention, the correction 
25 command is input to the signal processing apparatus by 
receiving of a broadcast signal or a communication signal ♦ 

In one embodiment of the invention, the correction 
command is recorded on a recording medium and is input to 
30 the signal processing apparatus by reproduction of the 
recording medium. 

Due to such a structure, the correction command can 
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be input to the signal processing apparatus by reproducing 
the data recorded on the recording medium. 



In one embodiment of the invention, the memory is 
5 arranged so as to receive at least one filter coefficient 
for correcting the acoustic signal from outside the signal 
processing apparatus, and to add the at least one filter 
coefficient received to the plurality of filter 
coefficients stored in the memory or to replace at least 
10 one of the plurality of filter coefficients stored in the 
memory with the at least one filter coefficient received. 

Due to such a structure, the content of the filter 
coefficients stored in the memory can be easily updated. 



15 



20 



In one embodiment of the invention, the at least one 
filter coefficient received is recorded on a recording 
medium and is input to the signal processing apparatus by 
reproduction of the recording medium. 

Due to such a structure, at least one filter 
coefficient can be input to the signal processing apparatus 
by reproducing the data recorded on the recording medium. 



25 In one embodiment of the invention, the signal 

processing apparatus further includes a buffer memory for 
temporarily accumulating the image signal and the acoustic 
signal. A speed at which the image signal and the acoustic 
signal are input to the buffer memory is higher than a speed 

30 at which the image signal and the acoustic signal are output 
from the buffer memory. The at least one filter coefficient 
recorded on the recording medium is stored in the memory 
while the image signal and the acoustic signal are output 
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from the buffer memory . A time period required for the image 
signal and the acoustic signal to be output from the buffer 
memory is equal to or longer than a time period for the at 
least one filter coefficient to be stored in the memory. 

Due to such a structure, acoustic signal correction 
data recorded on a recording medium can be reproduced without 
interrupting the image signal or the acoustic signal which 
is output from the reproduction apparatus* 

In one embodiment of the invention, the at least one 
filter coefficient selected includes at least one filter 
coefficient representing a transfer function showing an 
acoustic characteristic of a direct sound from a sound source 
to a viewer/listener. The correction section includes a 
transfer function correction circuit for correcting a 
transfer function of the acoustic signal in accordance with 
the at least one filter coefficient representing the 
transfer function. 

Due to such a structure, the viewer/ listener can 
perceive the virtual sound source using the speaker or the 
headphones . 

In one embodiment of the invention, the at least one 
filter coefficient selected includes at least one filter 
coefficient representing a transfer function showing an 
acoustic characteristic of a direct sound from a sound source 
to a viewer/ listener and at least one filter coefficient 
representing a reflection structure showing an acoustic 
characteristic of a reflection from the sound source to the 
viewer /listener . The correction section includes a 
transfer function correction circuit for correcting the 
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transfer function of the acoustic signal in accordance with 
the at least one filter coefficient representing the 
transfer function, a reflection addition circuit for adding 
a reflection to the acoustic signal in accordance with the 
5 at least one filter coefficient representing the reflection 
structure/ and an adder for adding an output from the 
transfer function correction circuit and an output from the 
reflection addition circuit. 

10 Due to such a structure, the viewer/listener can 

,5 perceive the virtual sound source using the speaker or the 

%0 headphones and requiring only a smaller calculation amount . 

H In one embodiment of the invention, the at least one 

17 15 filter coefficient selected includes at least one filter 

* coefficient representing a transfer function showing an 
% acoustic characteristic of a direct sound from a sound source 
11 to a viewer/listener and at least one filter coefficient 
i: representing a reflection structure showing an acoustic 

* 20 characteristic of a reflection from the sound source to the 

viewer/listener. The correction section includes a 
transfer function correction circuit for correcting the 
transfer function of the acoustic signal in accordance with 
the at least one filter coefficient representing the 
25 transfer function, and a reflection addition circuit for 
adding a reflection to an output of the transfer function 
correction circuit in accordance with the at least one filter 
coefficient representing the reflection structure. 

30 Due to such a structure, the viewer/ listener can 

more clearly perceive the virtual sound source using the 
speaker or the headphones and using a smaller calculation 
amount. 
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In one embodiment of the invention, the filter 
coefficient selection section includes an automatic 
selection section for automatically selecting at least one 
of the plurality of filter coefficients stored in the memory 
based on the correction command, and a manual selection 
section for manually selecting at least one of the plurality 
of filter coefficients stored in the memory. 

Due to such a structure, the viewer/ listener can 
select automatic selection of a filter coefficient or manual 
selection of a filter coefficient. 

In one embodiment of the invention, the at least one 
filter coefficient representing the reflection structure 
includes a first filter coefficient representing a 
reflection structure showing an acoustic characteristic of 
a reflection from the sound source to the viewer/ listener 
when a distance between the sound source and the 
viewer/listener is a first distance, and a second filter 
coefficient representing a reflection structure showing an 
acoustic characteristic of a reflection from the sound 
source to the viewer/listener when the distance between the 
sound source and the viewer/ listener is a second distance 
which is different from the first distance* 

Due to such a structure, the distance between the 
virtual sound source and the viewer/ listener can be 
arbitrarily set. 

In one embodiment of the invention, the at least one 
filter coefficient representing the reflection structure 
includes a third filter coefficient representing a 
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reflection structure showing an acoustic characteristic of 
a reflection reaching the viewer/listener from a direction 
in a predetermined range. 

Due to such a structure, the sound field desired by 
the viewer/ listener can be provided at a higher level of 
precision. 

In one embodiment of the invention, the 
predetermined range is defined by a first straight line 
connecting the sound source and a center of a head of the 
viewer /listener and a second straight line extending from 
the center of the head of the viewer/listener at an angle 
of 15 degrees or less from the first straight line. 

Due to such a structure, the sound field desired by 
the viewer/ listener can be provided at a higher level of 
precision. 

In one embodiment of the invention, the acoustic 
signal includes multiple -channel acoustic signals, and the 
filter coefficient selection section selects a filter 
coefficient corresponding to each of the multiple -channel 
acoustic signals . 

Due to such a structure, the location of the virtual 
sound source desired by the viewer /listener can be realized. 

In one embodiment of the invention, the signal 
processing apparatus further includes a display section for 
displaying a distance between a sound source and a 
viewer/listener. 
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Due to such a structure, the viewer/listener can 
visually perceive the distance between the virtual sound 
source and the viewer/listener * 

According to another aspect of the invention, a 
recording medium includes an acoustic data area for storing 
an acoustic signal; an image data area for storing an image 
signal; a navigation data area for storing navigation data 
showing locations of the acoustic data area and the image 
data area; and an assisting data area for storing assisting 
data. Acoustic signal correction data is stored in at least 
one of the acoustic data area, the image data area, the 
navigation data area, and the assisting data area. The 
acoustic signal correction data includes at least one of 
a correction command for specifying a correction method for 
the acoustic signal and a filter coefficient for correcting 
the acoustic signal. 

Due to such a structure, the acoustic signal can be 
corrected in association with reproduction of an image 
signal or an acoustic signal stored on the recording medium. 

In one embodiment of the invention, the correction 
command is stored in at least one of the acoustic data area, 
the image data area, and the navigation data area, and the 
filter coefficient is stored in the assisting data area. 

Due to such a structure, reproduction of the image 
signal, the acoustic signal or the navigation data is 
prevented from being interrupted by reproduction of a filter 
coefficient requiring a larger capacity than the correction 
command . 
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In one embodiment of the invention, the image data 
area stores at least one image pack, and the image pack 
includes the image signal and the acoustic signal correction 
data* 

Due to such a structure, the correction method for 
the acoustic signal can be changed in accordance with the 
change in the image signal. 

In one embodiment of the invention, the acoustic 
data area stores at least one acoustic pack, and the acoustic 
pack includes the acoustic signal and the acoustic signal 
correct ion dat a . 

Due to such a structure, the correction method for 
the acoustic signal can be changed in accordance with the 
change in the acoustic signal. 

In one embodiment of the invention, the navigation 
data area stores at least one navigation pack, and the 
navigation pack includes the navigation data and the 
acoustic signal correction data. 

Due to such a structure, the correction method for 
the acoustic signal can be changed in accordance with the 
change in the image signal or the acoustic signal which 
changes based on the navigation data. 

Thus , the invention described herein makes possible 
the advantages of providing a signal processing apparatus 
of an acoustic signal for reproducing an image signal and 
an acoustic signal while fulfilling various requests from 
viewers /listeners, and a recording medium having such an 
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image signal and an acoustic signal recorded thereon. 

These and other advantages of the present invention 
will become apparent to those skilled in the art upon reading 
and understanding the following detailed description with 
reference to the accompanying figures. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1A is a block diagram illustrating a 
structure of a signal processing apparatus la according to 
one example of the present invention; 

Figure IB is a block diagram illustrating another 
form of using the signal processing apparatus la according 
to the example of the present invention; 

Figure 1C is a block diagram illustrating still 
another form of using the signal processing apparatus la 
according to the example of the present invention; 

Figure 2 shows an example of a logic format of a DVD 

1; 

Figure 3 shows an example of a logic format of a still 
picture data area 14 shown in Figure 2; 

Figure 4 shows an example of a logic format of an 
acoustic data area 15 shown in Figure 2; 

Figure 5 shows another example of the logic format 
of the DVD 1; 
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Figure 6 shows an example of a logic format of an 
image /acoustic data area 54 shown in Figure 5; 

Figure 7 shows an example of a correction command 
and a filter coefficient; 

Figure 8A shows a state in which a signal recorded 
on the DVD 1 is reproduced; 

Figure 8B shows a state in which a signal recorded 
on the DVD 1 is reproduced; 

Figure 8C shows a state in which a signal recorded 
on the DVD 1 is reproduced; 

Figure 9A shows a state in which a signal recorded 
on the DVD 1 is reproduced; 

Figure 9B shows a state in which a signal recorded 
on the DVD 1 is reproduced; 

Figure 9C shows a state in which a signal recorded 
on the DVD 1 is reproduced; 

Figure 10A is a block diagram illustrating an 
exemplary structure of a correction section 5; 

Figure 10B is a block diagram illustrating another 
exemplary structure of the correction section 5; 



Figure IOC is a block diagram illustrating still 
another exemplary structure of the correction section 5; 
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Figure 11 is a plan view of a sound field 94; 

Figure 12 is a block diagram illustrating an 
exemplary structure of a transfer function correction 
circuit 91; 

Figure 13 is a block diagram illustrating another 
exemplary structure of the transfer function correction 
circuit 91; 

Figure 14 is a block diagram illustrating an 
exemplary structure of a reflection addition circuit 92; 

Figure 15 is a block diagram illustrating another 
exemplary structure of the reflection addition circuit 92; 

Figure 16 is a block diagram illustrating still 
another exemplary structure of the reflection addition 
circuit 92; 

Figure 17 is a block diagram illustrating still 
another exemplary structure of the reflection addition 
circuit 92; 

Figure 18 is a block diagram illustrating an 
exemplary structure of a filter coefficient selection 
section 3; 

Figures 19A, 19B and 19C show various types of a 
switch provided in a manual selection section 111; 

Figure 20A is a block diagram illustrating another 
exemplary structure of the filter coefficient selection 
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section 3; 

Figure 20B is a block diagram illustrating still 
another exemplary structure of the filter coefficient 
selection section 3; 

Figure 21A is a plan view of a sound field 122; 

Figure 21B is a side view of the sound field 122; 

Figure 22 shows reflection structures 123a through 
123n obtained at the position of the left ear of a 
viewer/listener 120; 

Figure 23 is a plan view of a sound field 127 in which 
five sound sources are provided; 

Figure 24 shows reflection structures respectively 
for directions from which sounds are transferred in areas 
126a through 126e in a reflection structure 123a; 

Figure 25 is a block diagram illustrating an 
exemplary structure of the correction section 5 for 
reproducing the sound field 122 using reflection structures 
128a through 128e; 

Figure 26 is a block diagram illustrating an 
exemplary structure of the correction section 5 for 
reproducing the sound field 122 using headphones 6? 

Figure 27 is a plan view of the sound field 127 
reproduced by the correction section 5 shown in Figure 26; 
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Figure 28 is a block diagram illustrating an 
exemplary structure of the correction section 5 in the case 
where 5.1-ch acoustic signals by Dolby Surround are input 
to the correction section 5; 

5 

Figure 29 shows an example of an area defining a 
direction from which a reflection is transferred; 

Figure 30 shows measurement results of head-related 
10 transfer functions from a sound source to the right ear of 
a subject; 

Figure 31 shows measurement results of head-related 
transfer functions from a sound source to the right ear of 
15 a different subject; 

Figure 32A shows another example of an area defining 
a direction from which a reflection is transferred; 

20 Figure 32B shows still another example of an area 

defining a direction from which a reflection is transferred; 

Figure 33 shows reflection structures 133a through 

133n; 



25 



30 



Figure 34 is a block diagram illustrating another 
exemplary structure of the correction section 5 in the case 
where 5.1-ch acoustic signals of Dolby Surround are input 
to the correction section 5; 

Figure 35 shows locations of five virtual sound 
sources 130a through 130e; and 
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Figure 36 shows examples of displaying a distance 
between a virtual sound source and a viewer/ listener . 

DESCRIPTION OF THE PREFERRED EMBODIMENTS 

Hereinafter, the present invention will be 
described by way of illustrative examples with reference 
to the accompanying drawings. The following example are 
only illustrative and not intended to limit the scope of 
the present invention. In the following description, a DVD 
will be described as an example of a recording medium on 
which an image signal and an acoustic signal are recorded. 
It should be noted, however, the recording medium used by 
the present invention is not limited to a DVD. Usable 
recording media also include any other type of recording 
media (for example, optical disks other than DVDs and hard 
disks in the computers) . In the following example, an image 
signal, an acoustic signal, or acoustic signal correction 
data recorded on a recording medium is reproduced, so as 
to input the image signal, the acoustic signal, or the 
acoustic signal correction data to a signal processing 
apparatus . The present invention is not limited to this . 
For example, broadcast or communication may be received so 
as to input an image signal, an acoustic signal or acoustic 
signal correction data to a signal processing apparatus. 

1. Structure of a signal processing apparatus la 

Figure 1A shows a signal processing apparatus la 
according to an example of the present invention. The 
signal processing apparatus la is connected to a 
reproduction apparatus 2 for reproducing information 
recorded on a DVD 1. 
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The DVD 1 has, for example, an acoustic signal AS, 
an image signal VS, navigation data, assisting data, and 
acoustic signal correction data recorded thereon. The 
acoustic signal correction data includes a correction 
command for specifying a correction method of the acoustic 
signal AS, and at least one filter coefficient for correcting 
the acoustic signal AS. Alternatively, the acoustic signal 
correction data may include only the correction command or 
only at least one filter coefficient. 

The correction data and the filter coefficient 
included in the acoustic signal correction data are input 
to the signal processing apparatus la by reproducing the 
information recorded on the DVD 1 using the reproduction 
apparatus 2. The format of the DVD 1 will be described in 
detail below with reference to Figures 2 through 6. 

The signal processing apparatus la includes a memory 
4 for storing a plurality of filter coefficients for 
correcting the acoustic signal AS, a filter coefficient 
selection section 3 for receiving the correction command 
from outside the signal processing apparatus la and 
selecting at least one of the plurality of filter 
coefficients stored in the memory 4 based on the correction 
command, and a correction section 5 for correcting the 
acoustic signal AS using the at least one filter coefficient 
selected by the filter coefficient selection section 3. 

The memory 4 is configured to receive at least one 
filter coefficient for correcting the acoustic signal AS 
from outside the signal processing apparatus la. The at 
least one filter coefficient input to the memory 4 is added 
to the plurality of filter coefficients stored in the memory 
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4 . Alternatively, the at least one filter coefficient input 
to the memory 4 may replace at least one of the plurality 
of filter coefficients stored in the memory 4. 

The acoustic signal AS corrected by the correction 
section 5 is output to headphones 6. The headphones 6 
convert the corrected acoustic signal AS to a sound and 
outputs the sound. The image signal VS output from the 
reproduction apparatus 2 is output to an image display 
apparatus 7 (for example, a TV). The image display 
apparatus 7 displays an image based on the image signal VS. 
Reference numeral 8 represents a viewer/listener who views 
the image displayed on the image display apparatus 7 while 
wearing the headphones 6 . 

Figure IB shows another form of using the signal 
processing apparatus la according to the example of the 
present invention. In Figure IB, identical elements 
previously discussed with respect to Figure 1A bear 
identical reference numerals and the detailed descriptions 
thereof will be omitted. In the example shown in Figure IB, 
the signal processing apparatus la is connected to a receiver 
2b for receiving a broadcast signal. The receiver 2b may 
be, for example, a set top box. 

The broadcast may be, for example, a digital TV 
broadcast . Alternatively, the broadcast may be a streaming 
broadcast through an arbitrary network such as, for example, 
the Internet. An image signal, an acoustic signal or 
acoustic signal correction data received through such a 
broadcast may be temporarily accumulated in a recording 
medium (not shown) such as, for example, a hard disk, and 
then the accumulated data may be input to the signal 
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processing apparatus la. 

Figure 1C shows still another form of using the 
signal processing apparatus la according to the example of 
the present invention. In Figure 1C, identical elements 
previously discussed with respect to Figure 1A bear 
identical reference numerals and the detailed descriptions 
thereof will be omitted. In the example shown in Figure 1C, 
the signal processing apparatus la is connected to a 
communication device 2c for receiving a communication 
signal. The communication device 2c may be, for example, 
a cellular phone for receiving a communication signal 
through a wireless communication path. Alternatively, the 
communication device 2c may be, for example, a modem for 
receiving a communication signal through a wired 
communication path. Such a wireless communication path or 
a wired communication path may be connected to the Internet . 
The signal processing apparatus la may temporarily 
accumulate an image signal, an acoustic signal or acoustic 
signal correction data in a recording medium (not shown) 
such as, for example, a hard disk, and then input the 
accumulated data to the signal processing apparatus la. 

Hereinafter, elements of the signal processing 
apparatus la will be described using, as an example, the 
case where the signal processing apparatus la is connected 
to the reproduction apparatus 2 for reproducing information 
recorded on the DVD 1 as shown in Figure 1A. The following 
description is applicable to the case where the signal 
processing apparatus la is used in the forms shown in Figures 
IB and 1C. For example, the logical format of the DVD 1 
described later is applicable to the logical format of the 
broadcast signal shown in Figure IB or the logical format 
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of the communication signal shown in Figure 1C. 

2. Logical format of the DVD 1 

Figure 2 shows an example of the logical format of 
the DVD 1. 

In the example shown in Figure 2, the DVD 1 includes 
a data information recording area 10 for recording a volume 
and a file structure of the DVD 1, and a multi-media data 
area 11 for recording multi-media data including still 
picture data. Acoustic signal correction data 12a is stored 
in an area other than the data information recording area 
10 and the multi-media data area 11. 

The multi-media data area 11 includes a navigation 
data area 13 for recording information on the entirety of 
the DVD 1, menu information common to the entirety of the 
DVD 1 or the like, a still picture data area 14 for recording 
data on a still picture, and an acoustic data area 15 for 
recording acoustic data. A detailed structure of the still 
picture data area 14 will be described below with reference 
to Figure 3 . A detailed structure of the acoustic data area 
15 will be described below with reference to Figure 4 . 

The navigation data area 13 includes an acoustic 
navigation area 16, a still picture navigation area 17, and 
an acoustic navigation assisting area 18. 

The acoustic navigation area 16 has acoustic 
navigation data 19 and acoustic signal correction data 12b 
stored therein. 

The still picture navigation area 17 has still 
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picture navigation data 20 and acoustic signal correction 
data 12c stored therein. 

The acoustic navigation assisting area 18 has 
acoustic navigation assisting data 21 and acoustic signal 
correction data 12d stored therein. 

In this manner, the acoustic signal correction data 
12b, 12c and 12d are stored in the DVD 1 so as to accompany 
the corresponding navigation data. 

Figure 3 shows an example of the logical format of 
the still picture data area 14 . 

The still picture data area 14 includes a still 
picture information area 22, a still picture object 
recording area 23, and a still picture information assisting 
area 24. 

The still picture information area 22 has still 
picture information data 25 and acoustic signal correction 
data 12e stored therein. 

The still picture object recording area 23 has at 
least one still picture set 26. Each still picture set 26 
includes at least one still picture object 27. Each still 
picture object 27 includes a still picture information pack 
28 and a still picture pack 29. The still picture pack 29 
includes still picture data 30 and acoustic signal 
correction data 12f . 

The still picture information assisting area 24 has 
still picture information assisting data 31 and acoustic 
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signal correction data 12g stored therein. 

In this manner, the acoustic signal correction data 
12e, 12f and 12g are stored in the DVD 1 so as to accompany 
the corresponding data on the still picture. 

Figure 4 shows an example of the logical format of 
the acoustic data area 15. 

The acoustic data area 15 includes an acoustic 
information area 32, an acoustic object recording area 33, 
and an acoustic information assisting area 34. 

The acoustic information area 32 has acoustic 
information data 35 and acoustic signal correction data 12h 
stored therein. 

The acoustic object recording area 33 has at least 
one acoustic object 36. Each acoustic object 36 includes 
at least one acoustic cell 37 . Each acoustic cell 37 
includes at least one acoustic pack 38 and at least one 
assisting information pack 39. The acoustic pack 38 
includes acoustic data 40 and acoustic signal correction 
data 121. The assisting inf ormation pack 39 includes 
assisting information data 41 and acoustic signal 
correction data 12 j. 

Each acoustic object 36 corresponds to at least one 
tune. Each acoustic cell 37 represents a minimum unit of 
the acoustic signal AS which can be reproduced and output 
by the reproduction apparatus 2. Each acoustic pack 38 
represents one-frame acoustic signal AS obtained by 
dividing the acoustic signal AS into frames of a periodic 
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predetermined time period. Each assisting information pack 
39 represents a parameter or a control command used for 
reproducing the acoustic signal AS. 

5 The acoustic information assisting area 34 has 

acoustic information assisting data 42 and acoustic signal 
correction data 12k stored therein. 

In this manner, the acoustic signal correction data 
10 12h, 121, 12 J and 12k are stored in the DVD 1 so as to accompany 
the corresponding acoustic data. 

Figure 5 shows another example of the logical format 
of the DVD 1. 

15 

In the example shown in Figure 5 , the DVD 1 includes 
a data information recording area 51 for recording a volume 
and a file structure of the DVD 1, and a multi-media data 
area 52 for recording multi-media data including moving 
20 picture data. Acoustic signal correction data 12a is stored 
in an area other than the data information recording area 
51 and the multi-media data area 52. 

The mult i -media data area 52 includes a navigation 
25 data area 53 for storing navigation data, and at least one 
image/acoustic data area 54 for recording image/acoustic 
data. A detailed structure of the image/ acoustic data area 
54 will be described below with reference to Figure 6. 

30 The navigation data represents information on the 

entirety of the DVD 1 and/ or menu information common to the 
entirety of the DVD 1 (location of acoustic data area and 
image data area) . The image signal and the video signal 
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change in accordance with the navigation data. 

The navigation data area 53 includes an 
image/acoustic navigation area 55, an image/acoustic object 
navigation area 56, and an image/acoustic navigation 
assisting area 57. 

The image /acoustic navigation area 55 has 
image /acoustic navigation data 58 and acoustic signal 
correction data 12m stored therein. 

The image/acoustic object navigation area 56 has 
image/ acoustic object navigation data 60 and acoustic 
signal correction data 12p stored therein. 

The image /acoustic navigation assisting area 57 has 
image /acoustic navigation assisting data 59 and acoustic 
signal correction data 12n stored therein. 

In this manner, the acoustic signal correction data 
12m, 12n and 12p are stored in the DVD 1 so as to accompany 
the corresponding navigation data. 

Figure 6 shows an example of the logical format of 
the image /acoustic data area 54. 

The image /acoustic data area 54 includes a control 
data area 61 for recording control data common to the 
entirety of the image/acoustic data area 54, an AV object 
set menu area 62 for a menu common to the entirety of the 
image /acoustic data area 54, an AV object recording area 
63, and a control data assisting area 64 for recording 
control assisting data common to the entirety of the 
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image/acoustic data area 54. 

The AV object recording area 63 has at least one AV 
object 65 stored therein. Each AV object 65 includes at 
least one AV cell 66. Each AV cell 66 includes at least one 
AV object unit 67. Each AV object unit 67 is obtained by 
time-division-multiplexing at least one of a navigation 
pack 68, an A pack 69, a V pack 70 and an SP pack 71. 

The navigation pack 68 includes navigation data 72 
having a pack structure and acoustic signal correction data 
12q. The A pack 69 includes acoustic data 73 having a pack 
structure and acoustic signal correction data 12r. The V 
pack 70 includes image data 74 having a pack structure and 
acoustic signal correction data 12s. The SP pack 71 
includes sub image data 75 having a pack structure and 
acoustic signal correction data 12t. 

Each AV object 65 represents one-track image signal 
VS and acoustic signal AS. A track is a unit of the image 
signal VS and the acoustic signal AS based on which the image 
signal VS and the acoustic signal AS are reproduced by the 
reproduction apparatus 2. Each AV cell 66 represents a 
minimum unit of the image signal VS and the acoustic signal 
AS which can be reproduced and output by the reproduction 
apparatus 2. 

In this manner, the acoustic signal correction data 
12q, 12r , 12s and 12t are stored in the DVD 1 so as to accompany 
the corresponding image /acoustic data. 

The acoustic signal correction data 12a (Figures 2 
and 5) is stored in an area on the DVD 1 different from the 
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area in which the image signal VS and the acoustic signal 
AS are stored. Accordingly, the acoustic signal correction 
data 12a can be output from the reproduction apparatus 2 
before the image signal VS and the acoustic signal AS are 
output from the reproduction apparatus 2. For example, a 
plurality of pieces of acoustic signal correction data 12a 
for correcting an acoustic characteristic of a plurality 
of type of headphones usable by the viewer/ listener 8 are 
stored on the DVD 1 beforehand. The acoustic signal 
correction data 12a which corresponds to the headphones 6 
actually used by the viewer/listener 8 is selected, and the 
acoustic signal AS is corrected using the selected acoustic 
signal correction data 12a. In this manner, the acoustic 
signal AS can be corrected in a suitable manner for the 
headphones 6 actually used by the viewer/listener 8. 
Similarly, acoustic signal correction data 12a for 
realizing an acoustic characteristic desired by the 
viewer/listener 8 may be stored on the DVD 1 beforehand. 

The acoustic signal correction data 12c (Figure 2) 
and acoustic signal correction data 12e through 12g (Figure 
3) are stored in the DVD 1 so as to accompany the 
corresponding data on the still picture. Accordingly, the 
acoustic signal correction data 12c and 12e through 12g can 
be read from the DVD 1 when the data on the still picture 
is read. As a result, the acoustic signal correction data 
12c and 12e through 12g can be output from the reproduction 
apparatus 2 in synchronization with the output of the image 
signal VS from the reproduction apparatus 2. Thus, the 
acoustic signal AS can be corrected in association with the 
content of the still picture displayed by the image display 
apparatus 7. 
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For example, when the image display apparatus 7 
displays a site where the acoustic signal AS was recorded 
(for example, a concert hall or an outdoor site), the 
acoustic signal AS can be corrected using the acoustic signal 
correction data 12c and 12e through 12g which reproduce the 
sound field of the site where the recording was performed. 
As a result, the viewer/ listener 8 can enjoy the acoustic 
characteristic matching the image. 

For example, when the image display apparatus 7 
displays a short range view or a distant view of a musical 
instrument or a singer, the acoustic signal AS can be 
corrected using the acoustic signal correction data 12c and 
12e through 12g which reproduce the distance between the 
sound source to the viewer/listener 8, As a result, the 
viewer/listener 8 can enjoy the acoustic characteristic 
matching the image. 

The acoustic signal AS may be recorded by the 
producer of the DVD 1 (content producer) on the DVD 1 so 
as to have an acoustic characteristic in accordance with 
the still picture (acoustic characteristic matching the 
image ) . Such recording is usually performed by the content 
producer in a mixing room or acoustic studio while adjusting 
the acoustic characteristic. In this case, the acoustic 
signal AS can be corrected using the acoustic signal 
correction data 12c and 12e through 12g which reproduce the 
sound field of the mixing room or the acoustic studio. As 
a result, the viewer /listener 8 can enjoy the acoustic 
characteristic adjusted by the content producer (acoustic 
characteristic matching the image). 

The acoustic signal correction data 12b and 12d 
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(Figure 2) and the acoustic signal correction data 12h 
through 12k (Figure 4) are stored on the DVD 1 so as to 
accompany the acoustic data. Accordingly, the acoustic 
signal correction data 12b, 12d and 12h through 12k can be 
output from the reproduction apparatus 2 in synchronization 
with the output of the acoustic signal AS from the 
reproduction apparatus 2. Thus, the acoustic signal AS can 
be corrected in association with the content of the acoustic 
signal AS. 

For example, the acoustic signal AS can be corrected 
in accordance with the tune or the lyrics of the tune. As 
a result , the viewer /listener 8 can enjoy a preferable 
acoustic characteristic. 

The acoustic signal correction data 12m, 12n, and 
12p through 12t (Figures 5 and 6) are stored on the DVD 1 
so as to accompany the data on the image including moving 
picture and acoustic data. Accordingly, the acoustic 
signal correction data 12m, 12n, and 12p through 12t can 
be output from the reproduction apparatus 2 in 
synchronization with the output of the image signal VS and 
the acoustic signal AS from the reproduction apparatus 2. 
Thus, the acoustic signal AS can be corrected in association 
with the content of the image (moving picture) displayed 
by the image display apparatus 7 and/or the content of the 
acoustic signal AS. As a result, the viewer/ listener 8 can 
enjoy the acoustic characteristic matching the image. 

3. Correction command and filter coefficient 

Figure 7 shows an example of a correction command 
and a filter coefficient included in the acoustic signal 
correction data (for example, the acoustic signal 
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correction data 12a shown in Figure 2). 

As shown in Figure 7, a correction command 81 is 
represented by, for example, 2 bits. In this case, the 
filter coefficient stored in the memory 4 can be specified 
in four different ways using the correction command 81. The 
correction command 81 can specify one filter coefficient 
or a plurality of filter coefficients. 

A filter coefficient 82 is, for example, any one of 
filter coefficients 83a through 83n or any one of filter 
coefficients 84a through 84n. 

The filter coefficients 83a through 83n each 
indicate an "impulse response" representing an acoustic 
transfer characteristic from the sound source of a 
predetermined sound field to a listening point (a transfer 
function showing an acoustic characteristic of a direct 
sound) . The filter coefficients 84a through 84n each 
indicate a "reflection structure" representing a level of 
the sound emitted from the sound source with respect to the 
time period required for the sound to reach the listening 
point in a predetermined sound field (an acoustic 
characteristic of a reflection) . 

Each of the plurality of filter coefficients stored 
in the memory 4 is any one of the filter coefficients 83a 
through 83n or any one of the filter coefficients 84a through 
84n. It is preferable that a plurality of filter 
coefficient of different types are stored, so that acoustic 
characteristics of various fields can be provided to the 
viewer/listener 8. 
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For example, using the filter coefficient 83a as the 
filter coefficient 82, a convolution calculation of the 
impulse response corresponding to the filter coefficient 
83a can be performed by the correction section 5 . As a result , 
the viewer/listener 8 can listen to a sound reproducing an 
acoustic characteristic from the sound source to the 
listening point of a predetermined sound field. Using the 
filter coefficient 84a as the filter coefficient 82, the 
viewer/listener 8 can listen to a sound reproducing a 
reflection structure of a predetermined sound field. 

In the case where the correction command 81 is 
represented by 2 bits, the capacity required to record the 
correction command 81 can be sufficiently small. 
Accordingly, the capacity of the DVD 1 is not excessively 
reduced even when the correction command 81 is recorded on 
the DVD 1. 

It is preferable that the correction command 81 is 
stored in at least one of the navigation data area 13 (Figure 
2) , the still picture data area 14 (Figure 2) and the acoustic 
data area 15 (Figure 2) and therefore, the filter coefficient 
82 is stored in an area other than the navigation data area 
13, the still picture data area 14 and the acoustic data 
area 15 (for example, the assisting data area) . In this case, 
reproduction of the image signal VS, the acoustic signal 
AS or the navigation data is prevented from being interrupted 
by reproduction of the filter coefficient 82 which requires 
a larger capacity than the correction command 81. 

As described above, the signal processing apparatus 
la according to the present invention allows the 
viewer/ listener 8 to listen to the sound which is matched 
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to the image displayed by the image display apparatus 7 
through the headphones 6. The correction performed on the 
acoustic signal AS by the correction section 5 changes in 
accordance with a change in the image signal VS and/or a 
change in the acoustic signal AS, As a result, the 
viewer/listener 8 does not notice any discrepancies in a 
relationship between the image and the sound. 

The filter coefficients in the memory 4 used for 
correcting the acoustic signal AS can be appropriately added, 
selected or altered. Accordingly, the acoustic 

characteristic can be corrected in accordance with the 
headphones 6 actually used by the viewer/ listener 8 or in 
accordance with individual body features of the 
viewer /listener 8 (for example, the features of the ears 
and face of the viewer/listener 8), in addition to the 
acoustic signal AS being corrected in accordance with the 
change in the image signal VS and/ or the change in the 
acoustic signal AS. 

The capacity required to record the correction 
command 81 which is used to select the filter coefficient 
in the memory 4 can be relatively small. Accordingly, the 
capacity of the DVD 1 is not excessively reduced even when 
the correction command 81 is recorded on the DVD 1. The time 
period required to read the correction command 81 from the 
DVD 1 can be relatively short. Accordingly, the filter 
coefficient can be frequently switched in accordance with 
a change in the image signal VS or a change in the acoustic 
signal AS. As a result, the manner of correcting the 
acoustic signal AS can be changed so as to better reflect 
the intent of the producer of the image signal VS and the 
acoustic signal AS (contents). 
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In the case where a broadcast signal or a 
communication signal is received and acoustic signal 
correction data included in the broadcast signal or the 
communication signal is input to the signal processing 
apparatus la, the correction command 81 is input to the 
signal processing apparatus la by receiving the broadcast 
signal or the communication signal. A band width required 
to broadcast or communicate the correction command 81 can 
be relatively small. Therefore, the band width for the 
broadcast signal or the communication signal is not 
excessively reduced even when the correction command 81 is 
broadcast or communicated. 

By correcting the acoustic signal AS based on the 
acoustic signal correction data recorded on the DVD 1 as 
described above, the viewer /listener 8 can obtain a 
favorable viewing and listening environment. 

In this example, still picture data and acoustic 
data are recorded on the DVD 1. Even when only the acoustic 
data is recorded on the DVD 1, the acoustic signal AS can 
be corrected in a similar manner. In this case, an effect 
similar to that described above is provided. 

In this example, a filter coefficient included in 
the acoustic signal correction data recorded on the DVD 1 
is stored in the memory 4. The filter coefficient may be 
stored in the memory 4 beforehand. Alternatively, a filter 
coefficient stored in a flexible disk or a semiconductor 
memory can be transferred to the memory 4. Still 
alternatively, the filter coefficient may be input to the 
memory 4 from the DVD 1 when necessary. In these cases also. 
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an effect similar to that described above is provided. 

In this example, the correction command is 
represented by 2 bits. The present invention is not limited 
5 to this. The bit length of correction commands may be 
increased or decreased in accordance with the type of the 
filter coefficient stored in the memory 4 and the capacity 
of the DVD 1. The correction command may be of any content 
so long as the correction command can specify the filter 
10 coefficient used for correcting the acoustic signal AS. In 
this case also, an effect similar to that described above 
is provided. 

In this example, a filter coefficient representing 
15 an impulse response and a filter coefficient representing 
a reflection structure are described as filter coefficients . 
Any other type of filter coefficient which has a structure 
for changing the acoustic characteristic is usable. In this 
case also, an effect similar to that described above is 
20 provided. A filter coefficient representing an impulse 
response and a filter coefficient representing a reflection 
structure can be used together. 

In this example, the corrected acoustic signal AS 
25 is output to the headphones 6 . The device to which corrected 
acoustic signal AS is output is not limited to the headphones 
6. The corrected acoustic signal AS may be output to any 
type of transducer (for example, a speaker) which has a 
function of converting the electric acoustic signal AS into 
30 a sound wave. In this case also, an effect similar to that 
described above is provided. 



4. Use of a buffer memory 87 
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In order to correct the acoustic signal AS without 
interrupting the output from the reproduction apparatus 2, 
the signal processing apparatus la (Figure 1A) preferably 
includes a buffer memory 87. Hereinafter, use of the buffer 
memory 87 will be described. 

Figures 8A through 8C and 9A through 9C each show 
a state where the image signal VS, the acoustic signal AS 
and acoustic signal correction data recorded on the DVD 1 
are reproduced by the reproduction apparatus 2. 

Figures 8A and 9A each show an initial state 
immediately after the reproduction of the data on the DVD 
1 is started. Figures 8B and 9B each show a state later than 
the state shown in Figures 8A and 9A. Figures 8C and 9C each 
show a state later than the state shown in Figures 8B and 
9B. 

In Figures 8A through 8C # reference numeral 85 
represents an initial data area which is first reproduced 
after the reproduction of the data on the DVD 1 is started. 
Reference numeral 86 represents a data area immediately 
subsequent to the initial data area 85. Reference numeral 
88 represents an area in which acoustic signal correction 
data 12 is recorded . 

Reference numeral 87 represents a buffer memory for 
temporarily accumulating data reproduced from the initial 
data area 85 and sequentially outputting the accumulated 
data. The buffer memory 87 is controlled so that the speed 
for inputting data to the buffer memory 87 is higher than 
the speed for outputting the data from the buffer memory 
87. For example , the speed for outputting the data from the 
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buffer memory 87 is a speed required to perform a usual 
reproduction (reproduction at the IX speed) of the image 
signal VS or the acoustic signal AS from the DVD 1. The speed 
for inputting the data to the buffer memory 87 is hither 
than the speed required to perform the usual reproduction 
(reproduction at the IX speed) of the image signal VS or 
the acoustic signal AS from the DVD 1. 

At least one filter coefficient included in the 
acoustic signal correction data recorded on the DVD 1 is 
stored in the memory 4 during the output of the image signal 
VS or the acoustic signal AS from the buffer memory 87. 

The time period required for outputting the image 
signal VS or the acoustic signal AS from the buffer memory 
87 is equal to or longer than the time period required for 
storing the at least one filter coefficient included in the 
acoustic signal correction data in the memory 4. 

In the initial state shown in Figure 8A, the data 
recorded in the initial data area 85 is reproduced at a higher 
speed than the speed required for the usual reproduction. 
As a result, the image signal VS and the acoustic signal 
AS are input to the buffer memory 87 at a higher speed than 
the speed required for the usual reproduction. The buffer 
memory 87 accumulates the output from the initial data area 
85, and also outputs the image signal VS and the acoustic 
signal AS accumulated in the buffer memory 87 at the speed 
required for the usual reproduction. 

Upon termination of the output of the data 12 from 
the initial data area 85, the initial state shown in Figure 
8A is transferred to the state shown in Figure 8B. 
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In the state shown in Figure 8B, the acoustic signal 
correction data 12 recorded in the area 88 is reproduced 
at a higher speed than the speed required for the usual 
reproduction* As a result, a filter coefficient included 
in the acoustic signal correction data 12 is output to the 
memory 4. The buffer memory 87 outputs the image signal VS 
and the acoustic signal AS accumulated in the buffer memory 
87 at the speed required for the usual reproduction. 

Upon termination of the output of the data from the 
*3 area 88, the state shown in Figure 8B is transferred to the 

J3 state shown in Figure 8C. 

H 15 In the state shown in Figure 8C, the data recorded 

■^z in the data area 86 immediately subsequent to the initial 

s" data area 85 is reproduced at the speed required for the 

^ usual reproduction. As a result, a correction command 

m recorded in the data area 86 is output to the filter 

2 20 coefficient selection section 3. In accordance with the 
jpl correction command, the filter coefficient selection 

section 3 outputs, to the memory 4, a signal specifying a 
filter coefficient to be selected among the plurality of 
filter coefficients stored in the memory 4. The memory 4 
25 outputs the filter coefficient specified by the filter 
coefficient selection section 3 to the correction section 
5. The correction section 5 corrects the acoustic signal 
AS using the filter coefficient output from the memory 4. 

30 In the initial state shown in Figure 9A, the data 

recorded in the initial data area 85 is reproduced at a higher 
speed than the speed required for the usual reproduction. 
As a result, the image signal VS and the acoustic signal 
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AS are input to the buffer memory 87 at a higher speed than 
the speed required for the usual reproduction. The buffer 
memory 87 accumulates the output from the initial data area 
85, and also outputs the image signal VS and the acoustic 
signal AS accumulated in the buffer memory 87 at the speed 
required for the usual reproduction. 

The correction command recorded in the initial data 
area 85 is output to the filter coefficient selection section 
3 via the buffer memory 87. In accordance with the 
correction command, the filter coefficient selection 
section 3 outputs, to the memory 4, a signal specifying a 
filter coefficient to be selected among the plurality of 
filter coefficients stored in the memory 4. The memory 4 
outputs the filter coefficient specified by the filter 
coefficient selection section 3 to the correction section 
5. The correction section 5 corrects the acoustic signal 
AS using the filter coefficient output from the memory 4. 
(Figure 9A does not show the processing performed after the 
correction command is output to the filter coefficient 
selection section 3 until the acoustic signal AS is 
corrected. ) Since no filter coefficient is reproduced from 
the initial data area 85, the acoustic signal AS recorded 
in the initial data area 85 is output without being corrected 
(or corrected using one of the plurality of filter 
coefficients stored in the memory 4 beforehand). 

Upon termination of the output of the data from the 
initial data area 85, the initial state shown in Figure 9A 
is transferred to the state shown in Figure 9B. 

In the state shown in Figure 9B, the acoustic signal 
correction data 12 recorded in the area 88 is reproduced 
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at a higher speed than the speed required for the usual 
reproduction. As a result, a filter coefficient included 
in the acoustic signal correction data 12 is output to the 
memory 4. The buffer memory 87 outputs the image signal VS 
and the acoustic signal AS accumulated in the buffer memory 
87 at the speed required for the usual reproduction and 
outputs the correction command to the filter coefficient 
selection section 3. 

Upon termination of the output of the acoustic 
signal correction data 12 from the area 88, the state shown 
in Figure 9B is transferred to the state shown in Figure 
9C* 

In the state shown in Figure 9C, the data recorded 
in the data area 86 immediately subsequent to the initial 
data area 85 is reproduced at the speed required for the 
usual reproduction . As a result , a correction command 
recorded in the data area 86 is output to the filter 
coefficient selection section 3. In accordance with the 
correction command, the filter coefficient selection 
section 3 outputs, to the memory 4, a signal specifying a 
filter coefficient to be selected among the plurality of 
filter coefficients stored in the memory 4, The memory 4 
outputs the filter coefficient specified by the filter 
coefficient selection section 3 to the correction section 
5. The correction section 5 corrects the acoustic signal 
AS using the filter coefficient output from the memory 4. 

By effectively using the buffer memory 87 as 
described above, the acoustic signal AS can be corrected 
based on the acoustic signal correction data 12 without 
interrupting the output of any of the image signal VS or 
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the acoustic signal AS from the reproduction apparatus 2. 

In this example, the acoustic signal AS recorded in 
the initial data area 85 is output without being corrected 
(or corrected using one of the plurality of filter 
coefficients stored in the memory 4 beforehand) . The 
initial data area 85 preferably stores an acoustic signal 
AS which does not need to be corrected. In the initial data 
area 85, an acoustic signal AS and an image signal VS on 
a title of the content (for example, a movie) of the DVD 
1 and/or an advertisement or the like provided by the content 
producer, for example, may be stored. 

In this example, after reproduction of the data on 
the DVD 1 is started, the data stored in the initial data 
area 85 is first reproduced. Alternatively, after 
reproduction of the data on the DVD 1 is started, the data 
stored in the area 88 having the acoustic signal correction 
data 12 recorded therein may be first reproduced. In this 
case also, the acoustic signal AS can be corrected based 
on the acoustic signal correction data 12 without 
interrupting the output of any of the image signal VS or 
the acoustic signal AS from the reproduction apparatus 2. 

In this example, image data and acoustic data are 
recorded in the initial data area 85 . Alternatively, either 
one of the image data or the acoustic data, or other data 
(for example, navigation data) may be recorded in the initial 
data area 85. In this case also, an effect similar to that 
described above is provided. 

5. Structure of the correction section 5 

Figure 10A shows an exemplary structure of the 
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correction section 5 (Figure 1A) . The correction section 
5 shown in Figure 10A includes a transfer function correction 
circuit 91 for correcting a transfer function of an acoustic 
signal AS in accordance with at least one filter coefficient 
which is output from the memory 4. 

In the following description, a sound wave in the 
space is transferred as shown in Figure 11. 

In Figure 11, reference numeral 94 represents a 
space forming a sound field, and reference numeral 95 
represents a sound source positioned at a predetermined 
position. CI represents a transfer characteristic of a 
direct sound from a virtual sound source 95 to the right 
ear of the viewer/ listener 8, C2 represents a transfer 
characteristic of a direct sound from the virtual sound 
source 95 to the left ear of the viewer/listener 8, Rl 
represents a transfer characteristic of a reflection from 
the virtual sound source 95 to the right ear of the 
viewer/listener 8, and R2 represents a transfer 
characteristic of a reflection from the virtual sound source 
95 to the left ear of the viewer/listener 8. 

Hereinafter, with reference to Figure 12, how to 
determine a filter coefficient of the transfer function 
correction circuit 91 is determined when the 
viewer/listener 8 receives the sound through the headphones 
6. 

Figure 12 shows an exemplary structure of the 
transfer function correction circuit 91. 

The transfer function correction circuit 91 
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includes an FIR filter 96a and an FIR filter 96b. The 
acoustic signal AS is input to the FIR filters 96a and 96b. 
An output from the FIR filter 96a is input to a right channel 
speaker 6a of the headphones 6 . An output from the FIR filter 
5 96b is input to a left channel speaker 6b of the headphones 
6. 

The case of reproducing the sound from the virtual 
sound source 95 by the headphones 6 will be described. A 

10 transfer function of the FIR filter 96a is Wl, a transfer 
function of the FIR filter 96b is W2, a transfer function 
from the right channel speaker 6a of the headphones 6 to 
the right ear of the viewer/listener 8 is Hrr , and a transfer 
function from the left channel speaker 6b of the headphones 

15 6 to the left ear of the viewer/ listener 8 is Hll. In this 
case, expression (1) is formed. 

(CI + Rl) = Wl-Hrr 

(C2 + R2) = W2-H11 expression (1) 

20 

By using Wl and W2 obtained by expression (1) 
respectively as the transfer functions of the FIR filters 
96a and 96b, the sound from the virtual sound source 95 can 
be reproduced by the headphones 6. In other words, while 
25 the sound is actually emitted from the headphones 6, the 
viewer/listener 8 can perceive the sound as if it was emitted 
from the virtual sound source 95* 

Based on expression (1), the transfer function Wl 
30 of the FIR filter 96a and the transfer function W2 of the 
FIR filter 96b are given by expression (2). 



Wl = (CI + Rl)/Hrr 
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W2 = (C2 + R2)/H11 expression (2) 

Hereinafter, with reference to Figure 13, how to 
determine a filter coefficient of the transfer function 
correction circuit 91 is determined when the 
viewer/listener 8 receives the sound through a speaker 97a 
and a speaker 97b. 

Figure 13 shows an exemplary structure of the 
transfer function correction circuit 91. 

The transfer function correction circuit 91 
includes an FIR filter 96a and an FIR filter 96b. The 
acoustic signal AS is input to the FIR filters 96a and 96b. 
An output from the FIR filter 96a is input to a right channel 
speaker 97a and converted into a sound wave by the speaker 
97a. An output from the FIR filter 96b is input to a left 
channel speaker 97b and converted into a sound wave by the 
speaker 97b. 

The case of reproducing the sound from the virtual 
sound source 95 by the speakers 97a and 97b will be described. 
A transfer function of the FIR filter 96a is XI, and a 
transfer function of the FIR filter 96b is X2. A transfer 
function from the speaker 97a to the right ear of the 
viewer/ listener 8 is Srr, and a transfer function from the 
speaker 97a to the left ear of the viewer/listener 8 is Sri. 
A transfer function from the speaker 97b to the right ear 
of the viewer/listener 8 is Sir, and a transfer function 
from the speaker 97b to the left ear of the viewer/listener 
8 is Sll. In this case, expression (3) is formed. 



(CI + Rl) = XI -Srr + X2*Slr 
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(C2 + R2) = XI -Sri + X2-S11 expression (3) 

By using XI and X2 obtained by expression (3) 
respectively as the transfer functions of the FIR filters 
96a and 96b, the sound from the virtual sound source 95 can 
be reproduced by the speakers 97a and 97b. In other words, 
while the sound is actually emitted from the speakers 97a 
and 97b, the viewer /listener 8 can perceive the sound as 
if it was emitted from the virtual sound source 95. 

Based on expression (3), the transfer function XI 
of the FIR filter 96a and the transfer function X2 of the 
FIR filter 96b are given by expression (4). 

XI = {S11-(C1 + Rl) - Slr-(C2 + R2)}/(Srr-Sll - Srl-Slr) 
X2 = {Srr-(C2 + R2) - Sri- (CI + Rl) }/ (Srr-Sll - Srl-Slr) 

expression (4) 

Figure 10B shows another exemplary structure of the 
correction section 5 (Figure 1A) . 

The correction section 5 shown in Figure 10B 
includes a transfer function correction circuit 91 for 
correcting a transfer function of an acoustic signal AS in 
accordance with at least one filter coefficient which is 
output from the memory 4, a reflection addition circuit 92 
for adding a reflection to the acoustic signal AS in 
accordance with at least one filter coefficient which is 
output from the memory 4, and an adder 93 for adding the 
output from the transfer function correction circuit 91 and 
the output from the reflection addition circuit 92. 

The transfer function correction circuit 91 has a 
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filter coefficient for reproducing a transfer 
characteristic of a direct sound from the virtual sound 
source 95 to the viewer/listener 8, The operation of the 
transfer function correction circuit 91 shown in Figure 10B 
5 is the same as that of the transfer function correction 
circuit 91 shown in Figure 10A except that (CI + Rl) and 
(C2 + R2) in expressions (1) through (4) are respectively 
replaced with CI and C2. Therefore, the operation of the 
transfer function correction circuit 91 will not be 
10 described in detail. 

The reflection addition circuit 92 has a filter 
coefficient for defining the level of the sound emitted from 
the virtual sound source 95 and reflected at least once with 
15 respect to the time period required for the sound to reach 
the viewer/ listener 8. 

Figure 14 shows an exemplary structure of the 
reflection addition circuit 92 . 

20 

As shown in Figure 14, the reflection addition 
circuit 92 includes frequency characteristic adjustment 
devices 98a through 98n for adjusting the frequency 
characteristic of the acoustic signal AS, delay devices 99a 

25 through 99n for delaying the outputs from the respective 
frequency characteristic adjustment devices 98a through 98n 
by predetermined time periods, level adjusters 100a through 
lOOn for performing gain adjustment of the outputs from the 
respective delay devices 99a through 99n, and an adder 101 

30 for adding the outputs from the level adjusters 100a through 
lOOn. The output from the adder 101 is an output from the 
reflection addition circuit 92. 
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The frequency characteristic adjustment devices 
98a through 98n adjust the frequency characteristic of the 
acoustic signal AS by varying the level of a certain 
frequency band component or performing low pass filtering 
or high pass filtering. 

In this manner, the reflection addition circuit 92 
generates n number of independent reflections from the 
acoustic signal AS* The transfer functions Rl and R2 of the 
reflection in a space 94 can be simulated by adjusting the 
frequency characteristic adjustment devices 98a through 98n, 
the delay devices 99a through 99n, and the level adjusters 
100a through lOOn. This means that a signal other than a 
direct sound can be realized by the reflection addition 
circuit 92- 

The transfer function correction circuit 91 shown 
in Figure 10B can have a smaller number of taps of the FIR 
filters 96a and 96b than those of Figure 10A. The reason 
for this is because the FIR filters 96a and 96b in Figure 
10B need to only represent the transfer characteristic of 
the direct sound among the sounds reaching from the virtual 
sound source 95 to the viewer/listener 8, unlike the case 
of Figure 10A. 

The calculation time of the reflection addition 
circuit 92 can be usually shorter than the calculation time 
period of the FIR f liters , which have a large number of taps . 
Hence, the structure in Figure 10B can reduce the calculation 
time as compared to the structure in Figure 10A. 

The frequency characteristic adjustment devices 
98a through 98n, the delay devices 99a through 99n and the 
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level adjusters 100a through lOOn need not be connected in 
the order shown in Figure 14. A similar effect is provided 
even when they are connected in a different order. 

5 The number of the frequency characteristic 

adjustment devices need not match the number of the 
reflections. For example, as shown in Figure 15, the 
reflection addition circuit 92 may include only one 
frequency characteristic adjustment device 98a. In this 

10 case, the frequency characteristic adjustment device 98a 
may correct a characteristic of the representative 
reflection (for example, a frequency characteristic 
required to generate a reflection having the largest gain) . 
Alternatively, as shown in Figure 16, by setting an average 

15 characteristic of a plurality of similar reflections, the 
number of the frequency characteristic adjustment devices 
can be reduced. 

Although not shown, a reflection can be generated 
20 only by the delay devices 99a through 99n and the level 
adjusters 100a through lOOn without using the frequency 
characteristic adjustment devices 98a through 98n. In this 
case, the precision of simulating the space 94 is lowered 
but still an effect similar to the above -de scribed effect 
25 is provided. 

In Figures 15 and 16, the delay devices 99a through 
99n and the level adjusters 100a through lOOn may be 
connected in an opposite order to the order shown . An effect 
30 similar to that described above is provided. 

Figure 10C shows still another exemplary structure 
of the correction section 5 (Figure 1A) . 
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The correction section 5 shown in Figure IOC 
includes a transfer function correction circuit 91 for 
correcting a transfer function of an acoustic signal AS in 
5 accordance with at least one filter coefficient which is 
output from the memory 4, and a reflection addition circuit 
92 connected to an output of the transfer function correction 
circuit 91 for adding a reflection to the output from the 
transfer function correction circuit 91 in accordance with 
10 at least one filter coefficient which is output from the 
memory 4 . 

If The transfer function correction circuit 91 has a 

4J- filter coefficient for reproducing a transfer 

15 characteristic of a direct sound from the virtual sound 
y. source 95 to the viewer/listener 8. The operation of the 

^ transfer function correction circuit 91 shown in Figure 10C 

% is the same as that of the transfer function correction 

ft! circuit 91 shown in Figure 10A except that (CI + Rl) and 

%^ 20 (C2 + R2) in expressions (1) through (4) are respectively 
|i replaced with CI and C2. Therefore, the operation of the 

transfer function correction circuit 91 will not he 

described in detail. 

25 The reflection addition circuit 92 has a filter 

coefficient for defining the level of the sound emitted from 
the virtual sound source 95 and reflected at least once with 
respect to the time period required for the sound to reach 
the viewer/ listener 8. 

30 

Figure 17 shows an exemplary structure of the 
reflection addition circuit 92. 
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The structure shown in Figure 17 is the same as that 
of Figure 14 except that the acoustic signal AS input to 
the reflection addition circuit 92 is input to the adder 
101. Identical elements previously discussed with respect 
to Figure 14 bear identical reference numerals and the 
detailed descriptions thereof will be omitted. 

The acoustic signal As is input to the frequency 
characteristic adjustment devices 98a through 98n and also 
input to the adder 101. By using the output from the adder 
101 as the output from the correction section 5, the sound 
from the virtual sound source 95 can be reproduced by the 
headphones 6 or the speakers 97a and 97b in a manner similar 
to that shown in Figures 10A and 10B. 

An input signal to the frequency characteristic 
adjustment devices 98a through 98n is an output signal from 
the transfer function correction circuit 91. Therefore, a 
reflection generated in consideration of the transfer 
characteristic of the direct sound from the virtual sound 
source 95 to the viewer /listener 8 is added. This is 
preferable for causing the viewer/listener 8 to perceive 
as if the sound they heard was emitted from the virtual sound 
source 95. 

The frequency characteristic adjustment devices 
98a through 98n, the delay devices 99a through 99n and the 
level adjusters 100a through lOOn need not be connected in 
the order shown in Figure 17. A similar effect is provided 
even when they are connected in a different order. 

The number of the frequency characteristic 
adjustment devices need not match the number of the 
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reflections. For example, as shown in Figure 15, the 
reflection addition circuit 92 may include only one 
frequency characteristic adjustment device 98a. In this 
case, the frequency characteristic adjustment device 98a 
may correct a characteristic of the representative 
reflection (for example, a frequency characteristic 
required to generate a reflection having the largest gain) . 
Alternatively, as shown in Figure 16, by setting an average 
characteristic of a plurality of similar reflections, the 
number of the frequency characteristic adjustment devices 
can be reduced. 

Although not shown, a reflection can be generated 
only by the delay devices 99a through 99n and the level 
adjusters 100a through lOOn without using the frequency 
characteristic adjustment devices 98a through 98n. In this 
case, the precision of simulating the space 94 is lowered 
but still an effect similar to the above- described effect 
is provided. 

In Figures 15 and 16, the delay devices 99a through 
99n and the level adjusters 100a through lOOn may be 
connected in an opposite order to the order shown. An effect 
similar to that described above is provided. 

In this example, there are two reflections Rl and 
R2. Even when there are more reflections, an effect similar 
to that described above is provided. 

In this example, there is only one virtual sound 
source 95. In the case where a plurality of virtual sound 
sources 95 are provided, the above -described processing is 
performed for each virtual sound source. Thus, while the 
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sound is actually emitted from the headphones 6 or from the 
speakers 97a and 97b, the viewer/listener 8 can perceive 
the sound as if it was emitted from the plurality of virtual 
sound sources 95* 

6. Structure of the filter coefficient selection section 
3 

Figure 18 shows an exemplary structure of the filter 
coefficient selection section 3 (Figure 1A) . 

As shown in Figure 18, the filter coefficient 
selection section 3 includes an automatic selection section 

110 for automatically selecting at least one of the plurality 
of filter coefficients stored in the memory 4, in accordance 
with a correction command, and a manual selection section 

111 for manually selecting at least one of the plurality 
of filter coefficients stored in the memory 4. 

The manual selection section 111 may include, for 
example, a plurality of push-button switches 112a through 
112n as shown in Figure 19A, a slidable switch 113 as shown 
in Figure 19B, or a rotary switch 114 as shown in Figure 
19C. By selecting a desired type of signal processing, the 
viewer/listener 8 can select at least one of the plurality 
of filter coefficients stored in the memory 4 . The selected 
filter coefficient is output to the correction section 5. 

The push-button switches 112a through 112n are 
preferable when the viewer/ listener 8 desires discontinuous 
signal processing (for example, when the viewer/ listener 
8 selects a desired concert hall to be reproduced in acoustic 
processing performed for providing an acoustic signal with 
an acoustic characteristic of a concert hall) . 
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The slidable switch 113 is preferable when the 
viewer/ listener 8 desires continuous signal processing (for 
example, when the viewer/listener 8 selects a desired 
position of the virtual sound source 95 to be reproduced 
in acoustic processing performed on an acoustic signal for 
causing the viewer/listener 8 to perceive as if the virtual 
sound source 95 was moved and thus the direction to the sound 
source and a distance between the sound source and the 
viewer/ listener 8 were changed) . 

The rotary switch 114 can be used similarly to the 
bush-button switches 112a through 112n when the selected 
filter coefficient changes discontinuously at every defined 
angle, and can be used similarly to the slidable switch 113 
when the selected filter coefficient changes continuously. 

The filter coefficient selection section 3 having 
the above-described structure provides the viewer/listener 
8 with a sound matching the image based on the correction 
command, and with a sound desired by the viewer/listener 
8. 

The structure of the filter coefficient selection 
section 3 is not limited to the structure shown in Figure 
18. Any structure which can appropriately select either 
signal processing desired by the viewer/ listener 8 and 
signal processing based on a correction command may be used. 
For example, the filter coefficient selection section 3 may 
have a structure shown in Figure 20A or a structure shown 
in Figure 20B. In the case of the structures shown in Figures 
20A and 20B, the manual selection section 111 is provided 
with a function of determining which of the selection results 
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has a higher priority among the selection result by the 
manual selection section 111 and the selection result by 
the automatic selection section 110. By selecting at least 
one of the plurality of filter coefficients stored in the 
memory 4 based on the determination result, an effect similar 
to that of the filter coefficient selection section 3 shown 
in Figure 18 is provided. 

7. Method for constructing a reflection structure 

Figure 21A is a plan view of a sound field 122, and 
Figure 21B is a side view of the sound field 122. 

As shown in Figures 21A and 21B, a sound source 121 
and a viewer/ listener 120 are located in the sound field 
122. In Figures 21A and 21B, Pa represents a direct sound 
from the sound source 121 which directly reaches the 
viewer /listener 120. Pb represents a reflection which 
reaches the viewer/listener 120 after being reflected by 
a floor. Pc represents a reflection which reaches the 
viewer/listener 120 after being reflected by a side wall. 
Pn represents a reflection which reaches the 
viewer /listener 120 after being reflected a plurality of 
times . 

Figure 22 shows reflection structures 123a through 
123n obtained at the position of the left ear of the 
viewer/listener 120 in the sound field 122. 

A sound emitted from the sound source 121 is divided 
into a direct sound Pa directly reaching the viewer/listener 
120 and reflections Pb through Pn reaching the 
viewer/listener 120 after being reflected by walls 
surrounding the sound field 122 (including the floor or side 
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walls ) . 

A time period required for the sound emitted from 
the sound source 121 to reach the viewer/listener 120 is 
in proportion to the length of the path of the sound. 
Therefore, in the sound field 122 shown in Figures 21A and 
2 IB, the sound reaches the viewer/ listener 120 in the order 
of the direct sound Pa, the reflection Pb, the reflection 
Pc and the reflection Pn. 

The reflection structure 123a shows the 
relationship between the levels of the direct sound Pa and 
the reflections Pb through Pn emitted from the sound source 
121 and the time periods required for the sounds Pa through 
Pn to reach the left ear of the viewer /listener 120. The 
vertical axis represents the level, and the horizontal axis 
represents the time. Time 0 represents the time when the 
sound is emitted from the sound source 121. Accordingly, 
the reflection structure 123a shows the sounds Pa through 
Pn in the order of reaching the left ear of the 
viewer /listener 120. Namely, the direct sound Pa is shown 
at a position closest to the time 0, and then the sound Pb, 
the sound Pc and the sound Pn are shown in this order. 
Regarding the level of the sound at the viewer/listener 120, 
the direct sound Pa is highest since the direct sound Pa 
is distance-attenuated least and is not reflection- 
attenuated. The reflections Pb through Pn are attenuated 
more as the length of the pass is longer and are also 
attenuated by being reflected. Therefore, the reflections 
Pb through Pn are shown with gradually lower levels. The 
reflection Pn has the lowest level among the reflections 
Pb through Pn. 
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As described above, the reflection structure 123a 
shows the relationship between the levels of the sounds 
emitted from the sound source 121 and the time periods 
required for the sounds to reach the left ear of the 
5 viewer/listener 120 in the sound field 122, In a similar 
manner, a reflection structure, showing the relationship 
between the levels of the sounds emitted from the sound 
source 121 and the time periods required for the sounds to 
reach the right ear of the viewer/ listener 120 in the sound 
10 field 122, can be obtained. By correcting the acoustic 
rg signal AS using the filter coefficient representing these 

^ reflection structures, the sound field 122 can be simulated. 

HP The reflection structures 123b through 123n show the 

Mr 

~n; 15 relationship between the levels of the direct sound Pa and 

N ; the reflections Pb through Pn emitted from the sound source 

JU 121 and the time periods required for the sounds Pa through 

tfl Pn to reach the left ear of the viewer/listener 120 when 

^; the distance from the sound source 121 to the viewer/ listener 

p 20 120 gradually increases. (Neither the direction nor the 

height of the sound source 121 with respect to the 

viewer/ listener 120 is changed. ) 



In the reflection structure 123b, the distance 
25 between the sound source 121 and the viewer/listener 120 
is longer than that of the reflection structure 123a. 
Therefore, the time period required for the direct sound 
Pa to reach the left ear of the viewer/listener 120 is longer 
in the reflection structure 123b than in the reflection 
30 structure 123a. Similarly, in the reflection structure 
123ii, the distance between the sound source 121 and the 
viewer/listener 120 is longer than that of the reflection 
structure 123b. Therefore, the time period required for the 
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direct sound Pa to reach the left ear of the viewer/listener 
120 is longer in the reflection structure 123n than in the 
reflection structure 123b. 

As the distance between the sound source 121 and the 
viewer/ listener 120 becomes longer, the amount of distance 
attenuation becomes larger. Therefore, the levels of the 
sounds Pa through Pn are lower in the reflection structure 
123b than in the reflection structure 123a. Similarly, the 
levels of the sounds Pa through Pn are lower in the reflection 
structure 123n than in the reflection structure 123b. 

The time periods required for the reflections Pb 
through Pn are also longer in the reflection structures 123b 
through 123n than in the reflection structure 123a. The 
level of the reflections Pb through Pn are lower in the 
reflection structures 123b through 123n than in the 
reflection structure 123a. However, in the reflection 
structures 123b through 123n, the reduction amount in the 
reflections Pb through Pn is smaller than the reduction 
amount in the direct sound Pa. The reason for this is as 
follows. Since the paths of the reflections Pb through Pn 
are longer than the path of the direct sound Pa, the ratio 
of the change in the path length due to the movement of the 
sound source 121 with respect to the total path length is 
smaller in the case of the reflections Pb through Pn than 
in the case of the direct sound Pa. 

As in the case of the reflection structure 123a, the 
reflection structures 123b through 123n show the 
relationship between the levels of the sounds emitted from 
the sound source 121 and the time periods required for the 
sounds to reach the left ear of the viewer/listener 120 in 
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the sound field 122. 

In a similar manner, reflection structures, showing 
the relationship between the levels of the sounds emitted 
from the sound source 121 and the time periods required for 
the sounds to reach the right ear of the viewer /listener 
120 in the sound field 122, can be obtained. By correcting 
the acoustic signal AS using the filter coefficient 
representing these reflection structures, the sound field 
122 can be simulated. 

In addition, by selecting the plurality of 
reflection structures 123a through 123n for use, the 
viewer /listener 120 can listen to the sound at the sound 
source at a position desired by the viewer/ listener 120 in 
the sound field 122. 

In the above example, there is only one sound source 
121 provided. When there are a plurality of sound sources 
also, the sound field can be simulated by obtaining the 
reflection structure in a similar manner. In the above 
example, the direction from which the sound is transferred 
is not defined for obtaining the reflection structure. The 
simulation precision of the sound field can be improved by 
obtaining the reflection structure while the direction from 
which the sound is transferred is defined. 

With reference to Figure 23, a method for 
constructing a reflection structure from a sound field 127 
including five speakers will be described. 

Figure 23 is a plan view of the sound field 127 in 
which five sound sources are located. 
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As shown in Figure 23, sound sources 125a through 
125e and a viewer/ listener 124 are located in the sound field 
127. The sound sources 125a through 125e are located so as 
to surround the viewer/ listener 124 at the same distance 
from the viewer/listener 124. In Figure 23, reference 
numerals 126a through 126e each represent an area (or a 
range) defined by lines dividing angles made by each two 
adjacent sound sources with the viewer/listener 124. 

The sound sources 125a through 125e are located so 
as to form a general small-scale surround sound source. The 
sound source 125a is for a center channel to be provided 
exactly in front of the viewer/listener 124. The sound 
source 125b is for a front right channel to be provided to 
the front right of the viewer/ listener 124. The sound 
source 125c is for a front left channel to be provided to 
the front left of the viewer/listener 124. The sound source 
125d is for a rear right channel to be provided to the rear 
right of the viewer/listener 124. The sound source 125e is 
for a rear left channel to be provided to the rear left of 
the viewer/ listener 124. 

The angle made by the sound sources 125a and 125b 
or 125o with the viewer/listener 124 is 30 degrees. The 
angle made by the sound sources 125a and 125d or 125e with 
the viewer/ listener 124 is 120 degrees. The sound sources 
125a through 125e are respectively located in the areas 126a 
through 126e. The area 126a expands to 30 degrees from the 
viewer/ listener 124. The areas 126b and 126c each expand 
to 60 degrees from the viewer/ listener 124. The areas 126d 
and 126e each expand to 105 degrees from the viewer/ listener 
124. 
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Hereinafter , an example of reproducing the sound 
field 122 shown in Figures 21A and 2 IB with the sound field 
127 will be described. In the sound field 122, the sound 
emitted from the sound source 121 reaches the 
viewer/listener 120 through various paths. Accordingly, 
the viewer/ listener 120 listen to the direct sound 
transferred from direction of the sound source 121 and 
reflections transferred in various directions . In order to 
reproduce such a sound field 122 with the sound field 127, 
a reflection structure representing the sound reaching the 
position of the left and right ears of the viewer/listener 
120 in the sound field 122 is obtained for each direction 
from which the sound is transferred, and the reflection 
structure is used for reproduction. 

Figure 24 shows reflection structures obtained for 
the direction from which the sound is transferred in each 
of the areas 126a through 126e. Reference numerals 128a 
through 128e respectively show the reflection structures 
obtained for the areas 126a through 126e. 

Figure 25 shows an exemplary structure of the 
correction section 5 for reproducing the sound field 122 
using the reflection structures 128a through 128e. 

The correction section 5 includes a transfer 
function correction circuit 91 and reflection addition 
circuits 92a though 92e. The transfer function correction 
circuit 91 is adjusted so that an acoustic characteristic 
of the sound emitted from the sound source 125a when reaching 
the viewer/ listener 124 is equal to the acoustic 
characteristic of the sound emitted from the sound source 
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121 when reaching the viewer/listener 120. The reflection 
addition circuits 92a through 92e are respectively adjusted 
so as to generate, from an input signal, reflections which 
have identical structures with the reflection structures 
128a through 128e and output the generated reflections. 

By inputting the outputs from the reflection 
addition circuits 92a through 92e to the sound sources 125a 
through 125e, the sound field 122 can be simulated at a higher 
level of precision. The reasons for this are because 
(i) the reflection structures 128a through 128e allow the 
levels of the reflections and the time periods required for 
the reflections to reach the viewer /listener 124 to be 
reproduced, and (ii) the sound source 125a through 125e 
allow the directions from which the reflections are 
transferred to be reproduced. 

Even when the transfer function correction circuit 
91 is eliminated from the structure shown in Figure 25, an 
effect similar to that described above is provided. The 
transfer function correction circuit 91 need not 
necessarily provided for the signal input to the sound source 
125a. 

In Figures 23 through 25, the sound field 122 is 
reproduced with the five sound source 125a through 125e. 
Five sound sources are not necessary required. For example, 
the sound field 122 can be reproduced using the headphones 
6. This will be described below. 

Figure 26 shows an exemplary structure of the 
correction section 5 for reproducing the sound field 122 
using the headphones 6. 
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As shown in Figure 26 , the correction section 5 
includes transfer function correction circuits 91a through 
91 j for correcting an acoustic characteristic of an acoustic 
signal AS, reflection addition circuits 92a through 92J 
respectively for adding reflections to the outputs from the 
transfer function correction circuits 91a through 91 j, an 
adder 129a for adding the outputs from the reflection 
addition circuits 92a through 92e, and an adder 129b for 
adding the outputs from the reflection addition circuits 
92f through 92j. The output from the adder 129a is input 
to the right channel speaker 6a of the headphones 6. The 
output from the adder 129b is input to the left channel 
speaker 6b of the headphones 6. In Figure 26, Wa through 
Wj represent transfer functions of the transfer function 
correction circuits 91a through 91j. 

Figure 27 shows a sound field 127 reproduced by the 
correction section 5 shown in Figure 26. Virtual sound 
sources 130a through 130e and a viewer/listener 124 are 
located in the sound field 127 . The positions of the virtual 
sound sources 130a through 130e are the same as the positions 
of the sound sources 125a through 125e shown in Figure 23. 

In Figure 27, Cr represents a transfer function from 
the sound source 125a to the right ear of the viewer/listener 
124 when the viewer/listener 124 does not wear the headphones 
6. CI represents a transfer function from the sound source 
125a to the left ear of the viewer/ listener 124 when the 
viewer/listener 124 does not wear the headphones 6. Hr 
represents a transfer function from the right channel 
speaker 6a of the headphones 6 to the right ear of the 
viewer/ listener 124. HI represents a transfer function 
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from the left channel speaker 6b of the headphones 6 to the 
left ear of the viewer/listener 124. 

The case of reproducing the sound from the sound 
source 125a by the headphones 6 will be described. Here, 
a transfer function of the transfer function correction 
circuit 91a is Wa, and a transfer function of the transfer 
function correction circuit 91f is Wf . A transfer function 
from the right channel speaker 6a of the headphones 6 to 
the right ear of the viewer/listener 124 is Hr , and a transfer 
function from the left channel speaker 6b of the headphones 
6 to the left ear of the viewer/listener 124 is HI. In this 
case, expression (5) is formed. 

Cr = Wa-Hr 

CI = Wf'Hl expression (5) 

By using Wa and Wf obtained from expression (5) 
respectively as the transfer functions of the transfer 
function correction circuits 91a and 9 If , the sound from 
the sound source 125a can be reproduced by the headphones 
6. Namely, while the sound is actually emitted from the 
headphones 6, the viewer/ listener 124 can perceive the sound 
as if it was emitted from the virtual sound source 125a. 

Based on expression (5), the transfer function Wa 
of the transfer function correction circuit 91a and the 
transfer function Wf of the transfer function correction 
circuit 91f are given by expression (6). 

Wa = Cr/Hr 

Wf = Cl/Hl expression (6) 
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The reflection addition circuit 92f adds, to the 
output from the transfer function correction circuit 9 If , 
a reflection having a reflection structure 128a obtained 
by extracting only a reflection from the direction of the 
range 126a represented by the sound source 125a to the left 
ear of the viewer/listener 124. Similarly, the reflection 
addition circuit 92a adds, to the output from the transfer 
function correction circuit 91a, a reflection having a 
reflection structure (not shown) obtained by extracting 
only a reflection from the direction of the range 126a 
represented by the sound source 125a to the right ear of 
the viewer/ listener 124 . The reflection structure obtained 
by extracting only the reflection reaching the right ear 
of the viewer/ listener 124 can be formed in a method similar 
to the method of obtaining the reflection structure 128a 
obtained by extracting only the reflection reaching the left 
ear of the viewer/ listener 124. As a result, the 
viewer/listener 124 perceives the presence of the virtual 
sound source 130a and also receives the sound accurately 
simulating the direct sound and the reflections from the 
sound source 125a through the headphones 6. 

Similarly, the case of reproducing the sound from 
the sound source 125b by the headphones 6 will be described. 
Here, a transfer function from the sound source 125b to the 
right ear of the viewer/listener 124 when the 
viewer/listener 124 does not wear the headphones 6 is Rr, 
and a transfer function from the sound source 125b to the 
left ear of the viewer/listener 124 when the viewer/ listener 
124 does not wear the headphones 6 is Rl. In this case, 
expression (7) is formed. 



Rr = Wb*Hr 
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Rl = Wg*Hl expression (7) 

By using Wb and Wg obtained from expression (7) 
respectively as the transfer functions of the transfer 
function correction circuits 91b and 91g, the sound from 
the sound source 125b can be reproduced by the headphones 
6. Namely, while the sound is actually emitted from the 
headphones 6, the viewer /listener 124 can perceive the sound 
as if it was emitted from the virtual sound source 125b. 

Based on expression (7), the transfer function Wb 
of the transfer function correction circuit 91b and the 
transfer function Wg of the transfer function correction 
circuit 91g are given by expression (8). 

Wb = Rr/Hr 

Wg = Rl/Hl expression (8) 

The reflection addition circuit 92g adds, to the 
output from the transfer function correction circuit 91g, 
a reflection having a reflection structure 128b obtained 
by extracting only a reflection from the direction of the 
range 126b represented by the sound source 125b to the left 
ear of the viewer/listener 124. Similarly, the reflection 
addition circuit 92b adds, to the output from the transfer 
function correction circuit 91b, a reflection having a 
reflection structure (not shown) obtained by extracting 
only a reflection from the direction of the range 126b 
represented by the sound source 125b to the right ear of 
the viewer/listener 124 . The reflection structure obtained 
by extracting only the reflection reaching the right ear 
of the viewer/ listener 124 can be formed in a method similar 
to the method of obtaining the reflection structure 128b 
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obtained by extracting only the reflection reaching the left 
ear of the viewer/listener 124. As a result, the 
viewer/ listener 124 perceives the presence of the virtual 
sound source 130b and also receives the sound accurately 
simulating the direct sound and the reflections from the 
sound source 125b through the headphones 6. 

Similarly, the viewer/listener 124 perceives the 
presence of the virtual sound source 130c by the transfer 
function correction circuits 91c and 91h and the reflection 
addition circuits 92c and 92h. The viewer /listener 124 
perceives the presence of the virtual sound source 130d by 
the transfer function correction circuits 91d and 911 and 
the reflection addition circuits 92d and 921. The 
viewer/ listener 124 perceives the presence of the virtual 
sound source 130e by the transfer function correction 
circuits 91e and 91J and the reflection addition circuits 
92e and 92j. 

As described above, the sound field 127 having the 
sound sources 125a through 125e located therein can be 
reproduced using the correction section 5 shown in Figure 
26. As a result, the sound field 122 which can be reproduced 
with the sound field 127 can also be reproduced. 

In this example, the sound is received using the 
headphones. The present invention is not limited to this. 
For example, even when the sound is received by a combination 
of two speakers, an effect similar to that described above 
is provided by combining the transfer function correction 
circuits and the reflection addition circuits. 

In this example, one acoustic signal is input to the 



- 66 - 



P26156 



correction section 5. The number of signals input to the 
correction section 5 is not limited to one. For example / 
an acoustic signal input to the correction section 5 can 
be 5 . 1- channel surround acoustic signals by Dolby Surround. 

The transfer function correction circuits 91a 
through 91 j and the reflection addition circuits 91a through 
923 need not be respectively connected in the order shown 
in Figure 26. Even when the transfer function correction 
circuits 91a through 913 and the reflection addition 
circuits 92a through 92j are respectively connected in an 
opposite order to the order shown in Figure 26, an effect 
similar to that described above is provided. 

Figure 28 shows an exemplary structure of the 
correction section 5 in the case where 5*l-ch acoustic 
signals by Dolby Surround are input to the correction section 
5. 

In the example shown in Figure 28, a center channel 
signal (Center) emitted from the sound source provided 
exactly in front of the viewer/ listener 124, a right channel 
signal (Front Right) provided to the front right of the 
viewer/listener 124, a left channel signal (Front Left) 
provided to the front left of the viewer/listener 124, a 
surround right channel signal (Surround Right) provided to 
the rear right of the viewer/listener 124, and a surround 
left channel signal (Surround Left) provided to the rear 
left of the viewer/listener 124 are input to the correction 
section 5. 

As shown in Figure 28, the signals input to the 
correction section 5 are corrected using the transfer 
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function correction circuits 91a through 91j and the 
reflection addition circuits 92a through 92 j . Thus, while 
the sound is actually emitted from the headphones 6, the 
viewer/listener 124 can perceive the sound as if it was 
multiple -channel signals emitted from the virtual sound 
sources 130a through 130e. 

The reflection structures used by the reflection 
addition circuits 92a through 92j are not limited to the 
reflection structures obtained in the sound field 122. For 
example, when a reflection structure obtained in a music 
hall desired by the viewer/listener 124 is used, favorable 
sounds can be provided to the viewer/listener 124. 

The acoustic signals input to the correction section 
5 are not limited to the center signal, the right channel 
signal, the left channel signal, the surround right signal, 
and the surround left signal. For example, a woofer channel 
signal, a surround back signal or other signals may be 
further input to the correction section 5. In this case, 
an effect similar to that described above is provided by 
correcting these signals using the transfer function 
correction circuits and the reflection addition circuits. 

In this example, an acoustic signal which is input 
to the correction section 5 is input to the transfer function 
correction circuits, and the output signals from the 
transfer function correction circuits are input to the 
reflection addition circuits. Alternatively, an acoustic 
signal which is input to the correction section 5 may be 
input to the reflection addition circuits, and the output 
signals from the reflection addition circuits may be input 
to the transfer function correction circuits. In this case 
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also, an effect similar to that described above is provided. 

The areas 126a through 126e defining the directions 
from which the reflections are transferred are not limited 
to the above-defined areas. The definition of the areas 
126a through 126e may be changed in accordance with the sound 
field or the content of the acoustic signal. 

For example, the area may be defined as shown in 
Figure 29. In Figure 29, line La connects the center 
position of the head of the viewer/listener 124 and the 
center position of a sound source 131 . Line Lb makes an angle 
of 6 degrees with line La. An area which is obtained by 
rotating line Lb axis -symmetrically with respect to line 
La (the hatched area in Figure 29) may define the direction 
from which the reflection is transferred when generating 
a reflection structure used by the reflection addition 
circuits. As the angle 0 made by line La and line Lb 
increases, more and more reflection components are included 
in the reflection structure, but the direction from which 
the reflection is transferred obtained by the transfer 
function correction circuits and the reflection addition 
circuits becomes different from the direction in the sound 
field to be simulated, resulting in the position of the 
virtual sound source becoming more ambiguous. As the angle 
6 made by line La and line Lb decreases, less and less 
reflection components are included in the reflection 
structure, but the direction from which the reflection is 
transferred obtained by the transfer function correction 
circuits and the reflection addition circuits becomes 
closer to the direction in the sound field to be simulated, 
resulting in the position of the virtual sound source 
becoming clearer. As the angle 6 made by line La and line 
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Lb, 15 degrees is preferable. The reason for this is because 
the features of the face and ears of the viewer/listener 
with respect to the sound changes in accordance with the 
direction from which the sound is transferred, and thus the 
characteristics of the sound received by the 
viewer/listener change. 

Figure 30 shows the results of measurement of a 
head-related transfer function from the sound source to the 
right ear of a subject. The measurement was performed in 
an anechoic chamber. In Figure 30, HRTF1 represents a 
head-related transfer function when one sound source is 
provided exactly in front of the subject. HRTF2 represents 
a head-related transfer function when one sound source is 
provided to the front left of the subject, at 15 degrees 
with respect to the direction exactly in front of the subject . 
HRTF3 represents a head-related transfer function when one 
sound source is provided to the front left of the subject, 
at 30 degrees with respect to the direction exactly in front 
of the subject. 

In Figure 30, the levels of the sounds are not very 
different in a frequency range of 1 kHz or lower. The 
difference between the levels of the sounds increases from 
1 kHz. Especially, HRTF1 and HRTF3 have a maximum of about 
10 dB. The difference between HRTF1 and HRTF2 is about 3 dB 
even at the maximum. 

Figure 31 shows the results of measurement of a 
head-related transfer function from the sound source to the 
right ear of a different subject . The measuring conditions , 
such as the position of the sound source and the like in 
Figure 31 are the same from those of Figure 30 except for 
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the subject* In Figure 31, HRTF4 represents a head-related 
transfer function when one sound source is provided exactly 
in front of the subject* HRTF5 represents a head-related 
transfer function when one sound source is provided to the 
front left of the subject, at 15 degrees with respect to 
the direction exactly in front of the subject. HRTF6 
represents a head-related transfer function when one sound 
source is provided to the front left of the subject, at 30 
degrees with respect to the direction exactly in front of 
the subject* 

A comparison between HRTF1 (Figure 30) and HRTF4 
(Figure 31), between HRTF2 (Figure 30) and HRTF5 (Figure 
31), and between HRTF3 (Figure 30) and HRTF6 (Figure 31) 
shows the following. The measurement results in Figures 30 
and 31 are not much different in a frequency range of about 
8 kHz (a deep dip) or lower, and the measurement results 
in Figures 30 and 31 are significantly different in a 
frequency range of above 8 kHz . This indicates that the 
characteristics of the subject greatly influences the 
head-related transfer function in the frequency range of 
above 8 kHz. In the frequency range of 8 kHz or lower, the 
head-related transfer functions of different subjects are 
similar so long as the direction of the sound source is the 
same. Therefore, when a sound field is simulated for a great 
variety of people in consideration of the direction from 
which the sound is transferred, using the transfer function 
correction circuits and the reflection addition circuits, 
the characteristics of the sound field can be simulated in 
the frequency range of 8 kHz or lower. In the frequency 
range of 8 kHz or lower, the head-related transfer function 
does not significantly change even when the direction of 
the sound source is different by 15 degrees. 
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When the angle 6 made by line La and line Lb in Figure 
29 is 15 degrees or less, the transfer function correction 
circuits are preferably adjusted so as to have a transfer 
function from the sound source 131 to the viewer/listener 
124, and the reflection addition circuits are preferably 
adjusted so as to have a reflection structure of a reflection 
transferred in the hatched area in Figure 29 . In this manner, 
a reflection structure including a larger number of 
reflections can be obtained despite that the position of 
the virtual sound source is clear. As a result, the 
simulation precision of the sound field is improved. 

In this example, each of the areas 126a through 126e, 
defining the direction from which the reflections are 
transferred, is obtained by rotating line Lb axis- 
symmetrically with respect to line La (the hatched area in 
Figure 29). In Figure 29, line La connects the center 
position of the head of the viewer/listener 124 and the 
center position of a sound source 131 . Line Lb makes an angle 
of 6 degrees with line La. Alternatively, each of the areas 
126a through 126e may be defined as shown in Figure 32A or 
Figure 32B. In Figure 32A, line La is a line extending from 
the right ear of the viewer/listener 124 in the forward 
direction of the viewer/listener 124. Line Lb makes an 
angle of 6 with line La. Each of the areas 126a through 126e 
may be defined as an area obtained by rotating line Lb 
axis -symmetrically with respect to line La (the hatched area 
in Figure 32A) . In Figure 32B, line La connects the right 
ear of the viewer/listener 124 and the center position of 
the sound source 131. Line Lb makes an angle of 9 with line 
La. Each of the areas 126a through 126e may be defined as 
an area obtained by rotating line Lb axis -symmetrically with 
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respect to line La (the hatched area in Figure 32B) . 

In this example, the method is described in which 
a plurality of reflection structures (for example, 
reflection structures 123a through 123n) are selectively 
used in order to provide the viewer/listener with the 
perception of distance desired by the viewer/ listener . The 
reflection structures need not be loyally obtained from the 
sound field to be simulated. For example, as shown in Figure 
33, the time axis of a reflection structure 132a for 
providing the perception of the shortest distance may be 
extended to form a reflection structure 132k or 132n for 
providing perception of a longer distance. Alternatively, 
the time axis of a reflection structure 133a for providing 
the perception of the longest distance may be divided or 
partially deleted based on a certain time width to form a 
reflection structure 133k or 133n for providing perception 
of a shorter distance. 

Figure 34 shows another exemplary structure of the 
correction section 5 in the case where 5.1-ch acoustic 
signals by Dolby Surround are input to the correction section 
5. In Figure 34, identical elements previously discussed 
with respect to Figure 28 bear identical reference numerals 
and the detailed descriptions thereof will be omitted. 

In the example shown in Figure 34, the correction 
section 5 includes adders 143a through 143e. The adders 
143a through 143e are respectively used to input the output 
from the reflection addition circuit 92a to the transfer 
function correction circuits 91a through 91e. The outputs 
from the transfer function correction circuits 91a through 
91e are added by the adder 129a. The output from the adder 
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129a is input to the right channel speaker 6a of the 
headphones 6. In the structure of the correction section 
5 shown in Figure 34, the reflection sound of the center 
signal reaching the viewer/ listener 124 from the directions 
of the virtual sound sources respectively represented by 
the transfer function correction circuits 91a through 91e 
is simulated at a significantly high level of precision. 

Figure 34 only shows elements for generating a 
signal to be input to the right channel speaker 6a of the 
headphones 6. A signal to be input to the left channel 
speaker of the headphones 6 can be generated in a similar 
manner. Figure 34 shows an exemplary structure for 
simulating the reflection of the center signal highly 
precisely. The correction section 5 may have a structure 
so as to simulate, to a high precision, the reflections of 
another signal (the front right signal, the front left signal, 
the surround right or the surround left signal) in a similar 
manner . 

The structure of the correction section 5 described 
in this example can perform different types of signal 
processing using the transfer function correction circuits 
and the reflection addition circuits , for each of a plurality 
of acoustic signals which are input to the correction section 
5 and/or for each of a plurality of virtual sound sources. 
As a result, as shown in Figure 35, a plurality of virtual 
sound sources 130a through 130e may be provided at desired 
positions . 

8 . Display of the distance between a virtual sound source 
and the viewer/ listener 

As described above, a virtual sound source is 
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created by signal processing performed by the correction 
section 5. By changing the filter coefficient used by the 
correction section 5, the distance between the virtual sound 
source and the viewer /listener can be controlled. 
Accordingly, by monitoring the change in the filter 
coefficient used by the correction section 5, the distance 
between the virtual sound source and the viewer/listener 
can be displayed to the viewer/ listener . 

Figure 36 shows examples of displaying the distance 
between the virtual sound source and the viewer/listener. 

A display section 141 includes lamps LEI through LE6 . 
The display section 141 causes one of the lamps corresponding 
to the distance between the virtual sound source and the 
viewer/listener to light up in association with the change 
in the filter coefficient used by the correction section 
5. Thus, the distance between the virtual sound source and 
the viewer/listener can be displayed to the 
viewer/ listener . 

A display section 142 includes a monitor M. The 
display section 142 numerically displays the distance 
between the virtual sound source and the viewer/ listener 
in association with the change in the filter coefficient 
used by the correction section 5, so as to display the 
distance to the viewer/listener. 

By providing the signal processing apparatus la 
(Figure 1A) with the display section 141 or 142, the 
viewer/ listener can visually perceive the distance between 
the virtual sound source and the viewer /listener as well 
as audibly* 
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In this example, the display section 141 includes 
six lamps. The number of lamps is not limited to six. The 
display section can display the distance between the virtual 
sound source and the viewer/listener in any form as long 
as the viewer/listener can perceive the distance. 

A signal processing apparatus according to the 
present invention allows the correction method of an 
acoustic signal to be changed in accordance with the change 
in an image signal or an acoustic signal. Thus, the 
viewer/listener can receive, through a speaker or a 
headphones, a sound matching an image now displayed by an 
image display apparatus. As a result, the viewer/ listener 
is prevented from experiencing an undesirable discrepancy 
in the relationship between the image and the sound. 

A signal processing apparatus according to the 
present invention allows the correction method of an 
acoustic signal to be changed in accordance with the acoustic 
characteristic of the speaker or the headphones used by the 
viewer/ listener or the acoustic characteristic based on the 
individual body features, for example, the shape of the ears 
and the face of the viewer/ listener . As a result, a more 
favorable listening environment can be provided to the 
viewer /listener. 

A signal processing apparatus according to the 
present invention prevents reproduction of an image signal, 
an acoustic signal or navigation data from being interrupted 
by reproduction of a filter coefficient requiring a larger 
capacity than the correction command. 
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A signal processing apparatus according to the 
present invention can reproduce acoustic signal correction 
data recorded on a recording medium without interrupting 
the image signal or the acoustic signal which is output from 
the reproduction apparatus. 

A signal processing apparatus according to the 
present invention can allow the viewer/ listener to perceive 
a plurality of virtual sound sources using a speaker or a 
headphones, and also can change the positions of the 
plurality of virtual sound sources. As a result, a sound 
field desired by the viewer /listener can be generated. 

A signal processing apparatus according to the 
present invention can display the distance between the 
virtual sound source and the viewer/listener to the 
viewer/listener. Thus, the viewer/listener can visually 
perceive the distance as well as audibly. 

Various other modifications will be apparent to and 
can be readily made by those skilled in the art without 
departing from the scope and spirit of this invention. 
Accordingly, it is not intended that the scope of the claims 
appended hereto be limited to the description as set forth 
herein, but rather that the claims be broadly construed. 



